## this folder contains python code for implementation of algorithms and numerical experiment in the main paper
### Logistic regression and Non-linear least square in deterministic setting
--------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------
## 1. Sparse_Init -- sparse layer implementation in Pytorch 
###          1.1. sparsedata.py -- sparse COO data creator
###          1.2. sparselinear.py -- Sparse (COO) layer implementation of fully connected layer
###          1.3. sparsemodule.py -- Sparse (COO) Module implementation of one-FC-layer model
###          1.4. sparseinit.py   -- Replicate Pytorch initialization from dense module/layer/parameters
###                               -- see https://pytorch.org/docs/stable/index.html for reference
###          1.5  sparsetrcg.py   -- Sparse (COO) implementation of Trust Region Newton-CG method
--------------------------------------------------------------------------------------------------------------------
## 2. Run - implementation of algorithm and code to execute runs
###          1. trcg_deterministic -- Trust Region Newton-CG run
###          2. oasis_deterministic -- OASIS run
###          3. adgd_deterministic -- Adaptive Gradient Descent run
####            Authors: ANONYMiZED, code was adopted from https://github.com/ymalitsky/adaptive_GD
###          4. adhess_deterministic -- AdaHessian run
--------------------------------------------------------------------------------------------------------------------
## 3. optim_adahessian.py - Authors: ANONYMiZED, code was downloaded from https://github.com/amirgholami/adahessian
## 4. oasis_optimizer.py - Authors: ANONYMiZED, code was adapted based on https://github.com/amirgholami/adahessian
--------------------------------------------------------------------------------------------------------------------
## To reproduce the results (as shown in Section 5 Experimental Results), please follow the instruction:
### 1. download datasets from LIBSVM at https://www.csie.ntu.edu.tw/~cjlin/libsvm/
### 2. specify a data folder, default '/data' 
### 3. go to '/Run' and open either one of the notebooks
### 4. run the chosen algorithms with default setting on saving log files. Specify a log folder, default '/Logs'
--------------------------------------------------------------------------------------------------------------------
## note.A: to find approximate solution for logistic regression, follow the above instruction and use 
##         'trcg_deterministic.ipynb'
--------------------------------------------------------------------------------------------------------------------
## note.B: all code are runable on GPU and/or CPU